Visual Interaction Networks
نویسندگان
چکیده
From just a glance, humans can make rich predictions about the future state of a wide range of physical systems. On the other hand, modern approaches from engineering, robotics, and graphics are often restricted to narrow domains and require direct measurements of the underlying states. We introduce the Visual Interaction Network, a general-purpose model for learning the dynamics of a physical system from raw visual observations. Our model consists of a perceptual front-end based on convolutional neural networks and a dynamics predictor based on interaction networks. Through joint training, the perceptual front-end learns to parse a dynamic visual scene into a set of factored latent object representations. The dynamics predictor learns to roll these states forward in time by computing their interactions and dynamics, producing a predicted physical trajectory of arbitrary length. We found that from just six input video frames the Visual Interaction Network can generate accurate future trajectories of hundreds of time steps on a wide range of physical systems. Our model can also be applied to scenes with invisible objects, inferring their future states from their effects on the visible objects, and can implicitly infer the unknown mass of objects. Our results demonstrate that the perceptual module and the object-based dynamics predictor module can induce factored latent representations that support accurate dynamical predictions. This work opens new opportunities for model-based decision-making and planning from raw sensory observations in complex physical environments.
منابع مشابه
Improving Energy-Efficient Target Coverage in Visual Sensor Networks
Target coverage is one of the important problems in visual sensor networks. The coverage should be accompanied with an efficient use of energy in order to increase the network lifetime. In this paper, we address the maximum lifetime for visual sensor networks (MLV) problem by maximizing the network lifetime while covering all the targets. For this purpose, we develop a simulated annealing (SA) ...
متن کاملConstruction and Analysis of Tissue-Specific Protein-Protein Interaction Networks in Humans
We have studied the changes in protein-protein interaction network of 38 different tissues of the human body. 123 gene expression samples from these tissues were used to construct human protein-protein interaction network. This network is then pruned using the gene expression samples of each tissue to construct different protein-protein interaction networks corresponding to different studied ti...
متن کاملComparison of Hubs in Effective Normal and Tumor Protein Interaction Networks
ABSTRACTIntroduction: Cancer is caused by genetic abnormalities, such as mutation of ontogenesis or tumor suppressor genes which alter downstream signaling pathways and protein-protein interactions. Comparison of protein interactions in cancerous and normal cells can be of help in mechanisms of disease diagnoses and treatments. Methods: We constructed protein interaction networks of cancerous a...
متن کاملAircraft Visual Identification by Neural Networks
In the present paper, an efficient method for three dimensional aircraft pattern recognition is introduced. In this method, a set of simple area based features extracted from silhouette of aerial vehicles are used to recognize an aircraft type from its optical or infrared images taken by a CCD camera or a FLIR sensor. These images can be taken from any direction and distance relative to the fly...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1706.01433 شماره
صفحات -
تاریخ انتشار 2017